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(54) Interactive artificial intelligence 

(57) Behavior of a device is modified based on the 
device's experience. The device includes: (i) a sensing 
unit for sensing signals; (ii) a concern-generating unit 
programmed to generate concern-parameters; (iii) an 
actuating unit for actuating the device; and (iv) a mem- 
ory storing a spreadsheet composed of multiple sets of 
data, each set corresponding to an event defined by sig- 
nals received by the device, variations in "its concern- 
parameters, and behavior of the device. When the 



device has variations in its concerns in a situation, the 
device extracts from the memory an event, if any, to best 
compensate for the variations to select behavior in the 
situation. The selected behavior may be adjusted by 
variations in its emotions. The behavior of the device is 
regulated by variations in its concerns, and the device 
can modify or improve "its behavior based on the stored 
information obtained in the past. 
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Description 

Background of the Invention 
5 Held of the Invention 

[0001 ] The present invention relates to a method for adjusting behavior of a device, and particularly to a method for 
adjusting the behavior based on the device's experience. 

10 Description of the Related Art 

[0002] Conventionally, various controlling methods have been available for controlling a device in accordance with 
a user's demand. 

[0003] In such controlling methods, normally, the user sets a target value at output which the user wants, and the 
is device is controlled in such a way that the output matches the target value, while feeding the output back to a control 
system which compares the feedback and the target value to adjust the output In the above, by feeding the output back 
to the system to adjust the output, the output of the device to be controlled can approach the target value, thereby 
achieving control satisfying the user's preference. 

[0004] However, the above control system may not be appropriate in an area where the user can develop emotions 
20 such as companionship toward a device. Such a device includes toys, games, communication tools, search tools, and 
other tools and devices subjected to personal use. Further, if a target value is not determinable due to the system's 
complexity or lack of information, the conventional control system does not work. 

[0005] Furthermore, if a device senses numerous pieces of information, it is difficult to sort them out and use them 
as useful information to act thereon. If the device is operated solely in accordance with the user's command, the above 
25 may not be a problem. However, for an autonomic device which acts without the user's command, it is important to 
obtain meaningful information. 

Summary of the Invention 

30 [0006] A general objective of the present invention is to provide a control system which enables a device to auton- 
omously modify its behavior or performance through interaction with its external environment, i.e., based on its experi- 
ence. That is a self-development system. The control system can generate pseudo-emotions which are used as a 
parameter for controlling behavior. The control system can collect information on an object of its own concern, and can 
store the information and update it by itself. This autonomous behavior system is advantageous, especially when 

35 applied to robots, toys, or games. The present invention has exploited a real-time basis behavior-adjusting system. 
[0007] An embodiment of the present invention is a method for adjusting behavior of a device based on the device's 
experience. The device comprises: (i) a sensing unit for sensing signals; (ii) a concern-generating unit programmed to 
generate concern-parameters; (iii) an actuating unit for actuating the device; and (iv) a memory storing a spreadsheet 
composed of multiple sets of data, each set corresponding to an event defined by signals received by the device, vari- 

40 ations in its concern-parameters, and behavior of the device. The method comprises steps of: (a) sensing signals 
receivable by the device; said device having concern-parameters (C } ) (i=entry number or time) which vary from prede- 
termined standards (SC); (b) extracting from the memory an event, if any, to best compensate for the variations in its 
concern-parameters; and (c) actuating the device based on the extracted information, and then computing variations in 
its concern-parameters (ACj), thereby updating the memory. Accordingly, when the device is in a situation, the device 

45 recalls (extracts) memory relevant to the situation based on variations in its concern-parameters. The behavior of the 
device is regulated by variations in its concern-parameters, and the device can modify or improve its behavior based on 
the information obtained in the past. 

[0008] In the above, in an embodiment, the device further comprises an emotion-generating unit programmed to 
generate emotion-parameters, and wherein, in step (a), the device has variations in its emotion-parameters, and in step 
so (c), behavior of the device extracted from the memory is modified by the variations in its emotion-parameters under pre- 
determined rules. By adjusting the behavior of the device based on variations in its emotion-parameters, the device can 
behave based on not only variations in its concern-parameters but also on current variations in its emotion-parameters, 
so that more complex and natural behavior of the device can be realized. 

[0009] In the above, in an embodiment, updated memory stores variations in its concern-parameters (ACj) linked 
55 with the event, wherein ACj = aACj + (1-a) AC' M (0<a<1) , wherein AC* M is the previous data saved under the 
index of the event Accordingly, the memory needs one column for one event and can reduce its capacity. Further, the 
influence of new experience can be controlled to moderate the behavior changes. 

[0010] In an embodiment, the relevancy of data stored in the memory can be determined as follows: The concern- 
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generating unit comprises a plurality (p) of concern modules, each differently programmed to generate and output the 
concern-parameters (C(n) j) (n=1 -p) , and, in step (b), the best compensating event is selected if the relevancy calcu- 
lated based on the difference between the initial variation (SC(n)-C(n)j) and the recorded variation (AC(n) M ) in each 
concern-parameter is hipher than a pre-selected score, wherein the relevancy is defined by 
5 E((SC(n)- C(n) j)-AC(n) M ) (n=1 -p) . The equation defining the relevancy is not limited to the above and can be of any 
type which is selected depending on the user's preference. In step (b), if no data on the event is extracted from the 
memory, the device can behave under predetermined rules. 

[001 1 ] Preferably, the control system further comprises a working memory which temporarily pools and stores infor- 
mation from the sensing unit and the concern-generating unit, in the first-mentioned memory until the device completes 
io its action, and which outputs information to the concern-generating unit, the actuating unit, and the first-mentioned 
memory. Accordingly, processing becomes efficient 

[0012] The concern-parameters and the emotion-parameters can have any labels and definitions. For example, the 
concern-parameters can represent "safety", "affection", "hunger", and "play", respectively, and the emotion-parameters 
can represent "happy", "angry", "surprised - , "sad", "fearful", and "disgust", respectively. The term "needs - can be a syn- 
15 onym for the concerns. The definition of each label can be predetermined. The number of concern-parameters and the 
number of emotion-parameters are not limited. The larger the" number of parameters, the more complex the behavior 
becomes. If an object-oriented language program, such as JAVA and C++, is used for parameter control, new labels 
can be automatically created, so that new behavior can be generated accordingly. 

[001 3] A device controlled by the above control system can behave autonomously and improve behavior based on 
20 its experience. 

[0014] The present invention can be applied to a method as well as an apparatus. Further, the system need not be 
an integrated system, but can be composed of plural separate units. That is, by networking plural devices or by using a 
separable medium (CD or IC card), an individual device can be downsized without losing memory and programming 
capacities. 

25 

Brief Description of the Drawings 
[0015] 

30 Figure 1 is a diagram showing an embodiment of the entire control system according to the present invention. 

Figure 2 is a diagram showing data flow and processing flow of an embodiment according to the present invention. 

Figure 3 is a flow chart diagram showing an embodiment of the control system according to the present invention. 

Figure 4 is a diagram showing an embodiment of the above metaphoric tanks in the present invention. 

Figure 5 is a diagram showing an embodiment of a flow chart showing time management flow in the present inven- 
ts tion. 

Figure 6 is a diagram showing an embodiment of a neural network for the concern-generating unit in the present 
invention. 

Figure 7 is a flow chart diagram showing an embodiment of data extraction processing in the present invention. 
Figure 8 is a diagram showing an embodiment of schematic processes of JAVA in the present invention. 
40 Figure 9 is a diagram showing an embodiment of an emotion-generating unit using a neural network in the present 
invention. 

Figure 10 is a diagram showing an embodiment of changes in emotions in the present invention. 
Figure 11 is a diagram showing an embodiment of data-storing processes under the rule, 
AC' n = aAC n + (1-a) AC'^ (0<a<1) in the present invention. 
45 Figure 12 is a diagram showing an embodiment of behavior patterns in the present invention. 

Figure 13 is a diagram showing an embodiment of a behavior control system in the present invention. 

Detailed Description of the Preferred Embodiment 

so [0016] The control system for controlling a device using emotion-parameters of the present invention (referred to as 
"control system') will be explained with reference to embodiments indicated in the figures. 

Basic Control System 

55 [0017] In an embodiment of the present invention, an agent or device comprises: (i) an external/internal condition 
recognition unit, (ii) a concern-generating unit, (Hi) an emotion-generating unit, (iv) a long-term memory, (v) a working 
memory, (vi) a behavior planning unit, and (vii) an actuating unit The above is a conceptual or functional structure, and 
need not be a physical structure; that is, these units need not be separately provided, but can be integrated. Further, 
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each unit can be disposed in a device, but can also be disposed separately from the device. 

[0018] Figure 1 is a diagram showing an embodiment of the entire control system according to the present inven- 
tion. As described above, this is a conceptual or functional structure, but not a physical structure. 
[0019] In the embodiment of Figure 1, an autonomic device comprises a s ensing unit 1, a recognition unit 2. a n 

5 emotion-generating unit 3, a concem^generating unit 4, a woriaag mpmnr y f or producing data sheets , a long-term 
memory 6 for storing data, a behavior planning unit 7, and an actuating unit 8. The sensing unit 1 senses various signals 
including external and internal information. External information includes visual, tactile, auditory information, and elec- 
trically transmitted signals, and internal information includes battery level and malfunction signals. The sensed signals 
are inputted into the recognition unit 2 which analyzes the signals to generate meaningful information which includes 

w characterization and identification of an event. This recognized information is inputted into the working memory 5 for 
producing data sheets. The concern-generating unit 4 is programmed to generate concern-parameters in response to 
the recognized Information. The concern-parameters are not limited and may include "safety", ■affection", "hunger", 
"study", and "play". The recognized information need not be all information but at least a piece of information (pre- 
selected) recognized at the recognition unit 2. Each concern-parameter is regulated independently of or harmoniously 

is with each other. Each concern-parameter may include a time factor; that is, the value of each concern-parameter 
changes with time (some of them fade with time while others increase with time). Further, each concern-parameter may 
have its own predetermined function and fluctuate accordingly. The concern-generating unit 4 receives a signal from 
. the working memory 5, which signal indicates whether or not the concern at issue is satisfied. The concern-generating 
unit 4 outputs a signal of concern to the working memory 5 for producing data sheets. The emotion-generating unit 3 is 

20 programmed to generate emotion-parameters in response to the recognized information and the generated concern- 
parameters. Thisr^c^q|ze^^ the inform ation {pre- selected) recognizedj atinejrffi- 
ogni tion_unit 2, and may be different from that used at the concern-generating unit 4 . The emotion-parameters are no t 
li mited but may include "happY*, "ang ry" t "surprised", "sad", fearful", and "disgust". Ea ch emotion-parame l^ 
I ated independently "of o r harmonbu^ Eac h ^^^'P^ 51116 ^ ma y Jndude a time fector^tfiat is . B| the 

25 valu e of each enrytion-^ram with timejso me of them iflade^ 

ther, tne emotion-parameters are generated when receiving information from the working memory 5 and signals from 
the concern-generating unit 4. The emotion-generating unit 3 outputs signals of emotions to the working memory 5 for 
producing data sheets. The long-term memory 6 is for storing data when the working memory 5 produces a final data 
sheet and inputs it to the long-term memory 6. The final data sheets are pooled and rearranged under the index of the 

30 event in the long-term memory 6 so that data can be retrieved as necessary. The data from the working memory 5 
updates the long-term memory 6. In an embodiment, the working memory 5 can be omitted, and the long-term memory 
6 can also function as the working memory 5, or vise versa. In the above, the term "event" means that, in a situation, 
the device observes an object and interacts with the object to modify its concern-parameters. In an embodiment, the 
woriking memory 5 can be omitted, and the long-term memory 6 can also function as the working memory 5, or vise 

35 versa. In another embodiment, the working memory 5 can be omitted, and each of the emotion-generating unit 3 and 
the concern-generating unit 4 can include a memory and can be connected to each other and to the other units. 
[0020] As described above, the working memory 5 receives signals from the recognition unit 2, the emotion-gener- 
ating unit 3, the concern-generating unit 4, and the long-term memory 6 to produce data sheets (algorithms will be 
explained later). In the working memory 5, a data sheet may indicate an event, which includes an object and current 

40 situation surrounding the device recognized by the recognition unit 2, current concerns, which are generated by the 
concern-generating unit 4, and current emotions, which are generated by the emotion-generating unit 3. Variations in 
the concern-parameters when previously encountering the event, which are extracted from the long-term memory 6, 
can be but need not be indicated in the data sheet in the working memory 5. Variations in the concern-parameters 
stored in the long-term memory 6 are effectively used for finding an event to best compensate for the current variations 

45 in the concern-parameters, so that the device can select an action which may be most appropriate in the situation to 
modify the concern-parameters. Based on these data, the behavior-planning unit 7 decides on a pattern of behavior 
and outputs a signal to the actuating unit 8. At the behavior planning unit 7, a behavior pattern may be modified based 
on current emotions which are obtained by the emotion-generating unit 3; that is, behavior is selected based on the data 
on the event extracted from the long-term memory 6, and is modified based on the current emotions. Patterns of behav- 

50 ior are not limited but may include "approach", "attack", "avoid", "explore", and "random". If the device has various mov- 
able parts such as a tail, ears, hands, and eyebrows, various patterns of behavior can be formed. In the above, for a 
simple device, behavior can be selected without referring to the current emotions, and the emotion-generating unit 3 
can be omitted. 

[0021] Upon action by the actuating unit 8, the result is sensed by the sensing unit 1 ; that is, the result is fed back 
55 to the device. For example, as a result of approaching the object, if the device was hit, that impact is sensed by the sens- 
ing unit 1 and inputted into the working memory 5 via the recognition unit 2 where the impact is converted to variations 
in its concerns. The data sheet for this event in the working memory 5 is complete upon inputting the variations in its 
concerns. This data sheet is provided to the long-term memory 6 and updates the data therein. After one data sheet is 
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complete (one event is over) and stored in the long-term memory 6, the device proceeds to a new event 
Basic Data Row 

5 [0022] Figure 2 is a schematic diagram showing data flow and processing flow of an embodiment according to the 
present invention. The processing order indicated in the figure is a simple suggestion and can be modified in the 
absence of material alteration. In this figure, first, the external/internal environment recognition unit (recognition unit) 2 
outputs information (e.g., CONTACT=TOUCH; IR (infrared sensor)=NONE; VISION=USER A, VOICE=NONE) to the 
working memory 5 which indicates the information on a data sheet 10a. The information is derived from the sensing unit 

10 1 and shows that the device is being touched, there is no obstacle in front of the device, the device is seeing user A, 
and the device is hearing no voice. The data sheet 1 0a includes items 'CONTACT, "IR", "VISION", "VOICE", "BEHAV- 
IOR", "CURRENT CONCERN", and "CURRENT EMOTION". These items are simply an example and can be selected 
depending on the types of sensors and recognition systems. In the data sheet 10a, "CONTACT", "IR", "VISION", and 
"VOICE" are updated using the information (current status) from the recognition unit 2. Based on information on 

is "VISION" and "VOICE", it is possible to determine the user's emotion. The "user" can be defined by motion, color, and 
voice, or simply by an electronic name tag. 

[0023] The current status is inputted into the concern-generating unit 4 and the emotion-generating unit 3. The con- 
cern-generating unit 4 is programmed to generate concern-parameters (explained later), and here, "play" (-0.3) has the 
highest intensity of its needs among the concern-parameters (e.g., "play", "safety", "hungry", etc.). The emotion-gener- 
20 ating unit 3 is programmed to generate emotion-parameters (explained later), and here, "happy" is zero. In the above, 
satisfaction standards for all of the concerns are zero, and negative values indicate insufficiency (i.e., the concern is in 
need) and positive values indicate sufficiency. 

[0024] Second, the working memory 5 includes the information (PLAY=-0.3, CURIOSITY=-0.1 , etc.) as "CURRENT 
CONCERN" provided by the concern-generating unit 4. 
25 [0025] Third, the working memory 5 includes the information (HAPPY=0.0, etc.) as "CURRENT EMOTION" pro- 
vided by the emotion-generating unit 3. 

[0026] Fourth, the information (CONTACT=TOUCH, IR=NONE, VISION=USER A, VOICE=NONE, PLAY=-0.3, 
CURIOSITY =-0.1 , etc.) is inputted to the long-term memory 6 to find an event which may best compensate for the cur- 
rent needs. The long-term memory 6 has a spreadsheet wherein data are stored under the indices of events. Here, the 

30 events are defined by "OBJECT, "USER EMOTION", "CONTACT, and "BEHAVIOR". The user's emotion can be 
determined simply by "VOICE", or by a facial expression recognition system (described later). "CONCERN VARIATION" 
is recorded under the indices of events to link variations in they concerns with the events, various data storage systems 
can be adapted in the long-term memory. Here, the memory stores data after every event. For example, in an event, 
when user A whose emotion appears to be neutral is touching the device, the device approaches user A, resulting in 

35 variations in the concerns, "PLAY" 0.3, "CURIOSITY" 0.1, "SAFETY" 0.1, etc. In another event, when user A who 
appears to be sad is touching the device, the device approaches user A, resulting in variations in the concerns, "PLAY" 
0.0, "CURIOSITY" 0.0, "SAFETY" -0.4, etc. 

[0027] Fifth, from the long-term memory 6, data stored under the index of the event are extracted. Extraction can 
be conducted in various ways. If the data sheet includes more information or items, data can be extracted under plural 

40 indices. For example, data can be collected under indices of "USER A", TOUCH", and "APPROACH", or an event can 
be defined by a single index, "USER A", and data can be collected under the index. The current emotion can be used 
as one of the indices to define each event In data sheet 1 0a, variations in the concerns are "PLAY -0.3", "CURIOSITY 
-0.1", etc. in a situation where the device is seeing user A, sensing no voice, being touched, and no obstacle is 
detected. Based on the information, the device can assume that the event is that the user A whose emotion appears to 

45 be neutral is touching the device, and the device needs "PLAY 0.3", "CURIOSITY 0.1 ", etc. to compensate for the insuf- 
ficiencies of the concerns. Under the index of this event, variations in the concerns which best compensate for the 
needs are searched for. Here, one event matching this situation is found and indicates that the device "approached" 
user A to obtain the variations in the concerns (BEHAVIOR=APPROACH). According to the recorded variations of the 
concerns for compensation, upon actuating the behavior (APPROACH), the device is expected to have "PLAY 0.0", 

so "CURIOSITY 0.0", etc. which satisfy the standards for concerns. These expected concerns can be inputted to the data 
sheet in the working memory. The searching method can be performed in various ways (explained later). 
[0028] Fifth, the information (BEHAVIOR=APP ROACH) is inputted into the working memory 5 which updates 
"BEHAVIOR" under predetermined rules (data sheet 10b). 

[0029] Sixth, the information (BEHAVIOR=APPROACH) is inputted into the behavior-planning unit 7 which updates 
55 "BEHAVIOR" under predetermined rules (data sheet 10b). Here, the information indicates that the device is being 
touched, there is no obstacle in front of the device, the device is seeing user A, and the emotion level (happy) of the 
device is zero (neutral). The behavior-planning unit 7 is programmed to determine a pattern of behavior based on these 
pieces of information. Here, the behavior-planning unit 7 selects "APPROACH" without modification by the emotion 



5 



EP1 083 488 A2 



level (since the level is zero), and the direction is toward user A. The approaching speed may be dependent on the emo- 
tion level (e.g., the higher the level, the higher the speed becomes). 

[0030] Seventh, the behavior-planning unit 7 outputs the information (BEHAVIOR=APPROACH, etc.) to the actuat- 
ing unit 8. The actuating unit 8 receives information from the behavior-planning unit 7 or directly from the working mem- 
5 ory 5, so that the actuating unit 8 can determine the direction of movement to achieve the task (APPROACH to USER 
A). Here, the actuating unit 8 determines 'FORWARD". As a result, the device further approaches user A while user A 
is touching the device. 

[0031] Eighth, the device receives a consequence of the action. Here, user A plays with the device by, e.g., patting 
the device in a predetermined way or talking to the device, to an extent greater than previously. 
io [0032] Ninth, as a result, the recognition unit 2 recognizes user A's reaction (this processing flow is not shown), and 
the emotion-generating unit 3 determines the emotion level as 'HAPPY - 0.2'. The emotion-generating unit 3 is pro- 
grammed to generate emotion-parameters. There are various ways to determine "CURRENT EMOTIONS" (explained 
later). 

[0033] Tenth, also, as a result, the recognition unit 2 recognizes user A's reaction (this processing flow is not 
15 shown), and the concern-generating unit 4 determines the concern levels as "PLAY 0.1", "CURIOSITY 0.0", etc. The 
concern-generating unit 4 is programmed to generate concern-parameters. There are various ways to determine "CUR- 
RENT CONCERN" (explained later). 

[0034] Eleventh, the current emotions in the working memory are updated (data sheet 1 0c). These emotion levels 
will affect subsequent behavior. 

zo [0035] Twelfth, the current concerns in the working memory are updated (data sheet 1 0c). 

[0036] Thirteenth, the difference between the initial concern levels (PLAY -0.3, CURIOSITY -0.1, etc.) and the 
updated concern levels (PLAY 0.1, CURIOSITY 0.0, etc.) is calculated (e.g., (PLAY 0.1)-(PLAY -0.3)=(PLAY 0.4), i.e., 
APLAY=-0.4), and the information including other data on the data sheet 10c ("CONTACT=TOUCH", "IR=NONE", 
"VISION=USER A", "VOICE=NONE", "APLAY=0.4", etc.) is transmitted to the long-term memory 6 and stored as new 

25 data under the indices of the event There are various ways to store the new data in the long-term memory 6 (explained 
later). 

[0037] As a result of updating the long-term memory 6, the device will behave differently than it did previously. Here, 
the event (when the user A whose emotion appears to be neutral is touching the device, the device approaches user A 
if there is no obstacle) may be extracted from the long-term memory 6 when the device is in higher need of "PLAY", etc. 
30 because the variation in "PLAY" (APLAY) for this event increases from 0.3 to 0.4. In the above, in order to moderate the 
influence of the instant occurrence (here, "APLAY 0.4"), various methods can be applied so that the influence of the 
recorded occurrence (here, "APLAY 0.3") can be saved to a certain degree, thereby avoiding a sudden change in 
behavior. The above methods will be explained later. 

[0038] In the above, the event can be retrieved solely under the index of the object (user A) or under fewer indices 
35 than the above. For example, if the event is retried under the index of the object, data stored under the index of the 
object in the long-term memory can be combined and averaged or processed under predetermined rules. There are 
various ways to select the best compensating event (explained later), and more complex behavior patters can be per- 
formed. That is, when the device is touched by user A and there is no obstacle, the device will approach user A at a low 
speed or will not move, depending on the manner of updating the long-term memory 6. 
40 [0039] In the figure, many flows are omitted. For example, the concern-generating unit 4 may receive the informa- 
tion (CONTACT=TOUCH, etc.) from the working memory 5 to generate the current concerns and to determine a satis- 
faction level of each concern. Further, the emotion-generating unit 3 may receive the information from the concern- 
generating unit 4 and/or the recognition unit 2 in order to generate emotion-parameters. 

45 Basic Flow Chart 

[0040] Figure 3 is a flow chart diagram showing an embodiment of the control system according to the present 
invention. As shown in Figure 3, the device senses the external and internal environment (step a). The control system 
generates current concerns by the concern-generating unit (step b) and generates current emotions by the emotion - 

so generating unit (step c). Based on the current concerns, variations in the concerns (deviations from the standards) to 
be compensated for are determined. Based on the variations in the concerns, the long-term memory is searched for an 
event relevant to the current event the device is now encountering (step d). The relevant event best compensates for 
the variations in the concerns. The control system checks if a relevant event is found (step e). If an event is found in the 
long-term memory, the control system extracts data on the event, especially behavior information (step f)- Based on the 

55 behavior information, candidate behavior is selected by the behavior-planning unit, which candidate behavior is modi- 
fied based on the current emotions (step g). The device is actuated based on the information from the behavior-plan- 
ning unit (step h). In step e, if no event is found in the long-term memory, the behavior-planning unit generates 
commands based on the current concerns and/or emotions under predetermined rules (step i). The device is then actu- 
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ated (step h). As a result, the device experiences consequences of the device's behavior. The consequence is sensed 
by the sensing unit and recognized by the recognizing unit, and then the data are recorded as a new event (if no previ- 
ous relevant event was found) or the data updates the previous data (if a relevant event was found). In the above, the 
emotion-generating unit is not indispensable, and step c can be omitted, because the control system can be operated 
5 simply by variations in the concerns. 

Sensing and Recognition Units 

[0041] In an embodiment, the device may comprise a CCD camera as a visual detection means, a pressure-sens- 
w ing sensor and an approach-sensing sensor as touch detection means, a microphone as a hearing-detection means, 
and an infrared sensor as an obstacle-detection means. The device may further comprise a battery capacity sensor 
which detects the remaining capacity of the battery. Alternatively, the device may comprise a radio wave signal receiver. 
By using these sensing means, the device detects objects, the environment, and internal conditions. Further, these sen- 
sors allow the device to detect the state of the user, such as a tone of voice, facial expressions, and gestures, and the 
is operational environments where the device is used. 

[0042] If the device is a robot, the CCD camera is installed on the top of the head and can be set in any direction 
via a universal joint For example, the robot can be controlled in such a way that the robot automatically moves toward 
an object, such as a human or animal, which is a cause or source of information such as changes in temperature and 
sound. Image information such as facial expressions of the user and surrounding environments is supplied to a control- 
20 ler. 

[0043] The pressure-sensing sensor may be installed in the lower front of the robot so that when the robot has 
actual contact with an obstacle, such information is provided to the controller. 

[0044] The microphone may be installed on the side of the head of the robot, and provides information to the con- 
troller upon collecting sound/voices arose around the robot. 

25 [0045] In the present invention, the sensed signals, i.e., primitive information, can be used directly, without further 
processing. For example, if a color is sensed, and if the control system is designed to work based on the color (an object 
is recognized simply by the color), no further processing is required. However, if the device is designed to respond to 
more than the color, the device needs to recognize more complex information and may require processing information. 
For example, based on the color, movement, and sound, the object can be recognized. Further, if complex recognition 

30 systems are used, the user's facial expression can be detected, and the device can respond to the emotions of the user 
which are represented by the facial changes (Hanaya, et al., "An attempt of individual identification from face photo- 
graphs", Technical Report of the Institute of Electronics, Information and Communication Engineers, CS96-123, IE96- 
92 (1996-12), pp. 55-60). A face neuron or neural network technology can be adapted. 

[0046] In the present invention, for the purpose of simple experimentation, by using a radio wave detection sensor, 
35 the device can detect an object which possesses an identification tag transmitting identification information. The radio 
wave detection sensor can transmit radio waves and receive resonance signals emitted from the identification tag which 
has a particular resonance circuit In addition, magnetic identification or bar code identification can be adapted. Further, 
a neural network can be used for recognizing an object based on the sensed signals. 

40 Concern-Generating Unit 

[0047] Tne concern-generating unit is used for selecting an object of concern, so that the device can behave and 
collect data efficiently, and can improve its behavior quickly, without information overflow. The device can select an 
object by itself, without a user's command, so that the device can behave autonomously. The concern-generating unit 

45 can be independent from the external conditions or environment, and can be a function of its own equations. However, 
the concern-generating unit may be affected by the internal conditions such as battery level and the degree of satisfac- 
tion of each concern-parameter. Further, the concern-generating unit may have a time factor and change each concern- 
parameter with time. A change with time may occur in the direction of fading. The functions controlling each concern- 
parameter can be independent of each other, and they may create fluctuation cycles of each concern-parameter. For 

so example, concern in "play" may fluctuate at predetermined intervals. The working memory may store data showing the 
number of occurrences of "play" and a degree of satisfaction, so that the concern-generating unit receives the data and 
modifies each concern-parameter. The working memory shown in Figure 3 can include the item "capacity" which shows 
a level or degree of satisfaction, and, for example, when the consequent emotion is positive, the working memory out- 
puts a signal to the concern-generating unit to raise the satisfaction level or degree. In general, the concerns are posi- 

55 tioned at a higher level than the emotions, and the emotions are controlled by the concerns. However, the emotions can 
be one of the factors that modify the concerns. In an embodiment, C = f (t, Cp) wherein C is a concern-parameter, f is 
a function, t is a time factor, and Cp is capacity. If each concern-parameter is metaphorically represented by contents 
in a tank, its satisfaction level can be represented by the level of the contents. Figure 4 is a diagram showing an embod- 



7 



EP 1 083 488 A2 



iment of the above metaphoric tanks. A tank 20 of "sleep" has a small hole 22 at the bottom, so that the contents 25 
(concern-parameter) leaks with time. The tank has two level switches; a first switch is a lower switch 24 disposed at a 
lower part of the tank, which triggers opening a valve 21 to fill the tank with the contents (concern-parameter), and the 
other switch is a higher switch 23 disposed at a higher part of the tank, which triggers dosing the valve 21 to stop filling 

5 the tank with the contents (concern-parameter). Even if the tank is full, the contents leak slowly with time, and eventually 
the lower switch will be activated. While the valve is open, new contents are introduced into the tank every time "sleep" 
is satisfied. The above mechanism can readily be realized in a program. If there are a plurality of concern-parameters, 
a plurality of tanks as above can be used, and the filling speed, the leaking speed, the positions of the lower switch and 
the higher switch can vary among the tanks (concern-parameters). In Rgure 4, a tank 26 of "hunger" is smaller than the 

10 tank 20 of "sleep". As described above, the concern-generating unit can readily be preprogrammed. 

[0048] In another embodiment, changes with time can be achieved as shown in Figure 5 which is a flow chart show- 
ing time management flow. In Rgure 5, if there is any signal stimulating "safety" (step i), the value of the concern-param- 
eter is modified in accordance with the intensity of the stimulation (step ii). If there is no signal stimulating "safety" in 
step i, the value of the concern-parameter is modified under predetermined rules (step iii). In Rgure 5, the value of the 

15 concern-parameter is fading or is multiplied by 0.9. 

[0049] In another embodiment, the concern-generating unit can be established using a neural network by regulat- 
ing a relationship between the recognized signals and the concern -parameters. Rgure 6 is a diagram showing an 
embodiment of a neural network 30 for the concern-generating unit When using the neural network, each concern- 
parameter is related to one another. In addition, output of the neural network is connected to a time management sub- 

20 routine 31 to change each output with time. The time management can be conducted by activating the time factor (e.g., 
reducing 1 0%) at pre-selected time intervals or after every (or pre-selected number) running cycle(s) of a program. 
[0050] The neural network can be established off-line or on-line. If it is conducted on-line, output of the device is fed 
back to the control system, and coupling coefficients can be adjusted. Further, coupling coefficients can be modified 
using evolutionary computing technology such as genetic algorithms and genetic programming. However, if on-line 

25 establishment is conducted, a training" period will be required. 

[0051] Further, the concern-generating unit can be constituted by a multi-dimensional map defined by the recog- 
nized signals and the concern-parameters. 

[0052] The concern-generating unit may select one concern-parameter which has the lowest value among all of the 
concern-parameters at the time the concern-generating unit is called for. Selection of a concern-parameter can be con- 
30 ducted in various ways, including selection at random or under predetermined rules or functions. 

Selecting and Recording Relevant Event 

[0053] Rgure 7 is a flow chart diagram showing an embodiment of selecting and recording an relevant event. As 
35 described above, the device determines variations in its concerns in a situation by the concern-generating unit (step 
S1). The situation is recognized by the recognition unit In Figure 2, the situation is that the device is seeing user A, 
healing no voice, being touched, and seeing no obstacle. This situation can be interpreted as follows: User A whose 
emotion appears to be neutral is touching the device surrounded by nothing. If the situation is defined by all of the 
above, a specific event can be found in the long-term memory. The relevancy of the event is determined first by deter- 
40 mining the relevancy of the situation (step S2). In an embodiment, the relevancy of the situation is determined on an 
element-by-element basis. In the above, if the situation is defined by all of the elements (e.g., user A, user's emotion, 
tactile sensation, obstacle), each element is compared with that of the current situation. In the above, some elements 
can be weighted heavily than others. For example, if "user A" is more important than "being touched", the element of 
"object" can be weighted more than "contact" by weighting the former with a factor. If there are plural elements (1 to p) 
45 of the situation at time i or data entry #i (S(n) $ ) (n=1 -p) , the relevant situation is selected from the long-term memory if 
the relevancy calculated based on the difference between the current situation (S(n)j) and the recorded situation (S(n)j_ 
j) (j<i) is higher than a pre-selected score, wherein the relevancy can be defined by, for example, the following: 

£(S(n)j-S(n) H ) 

50 

Z(S(n) r S(n) 4 ) 2 

S{Pi(S(1) r S(1) 4 )+ p 2 (S(2) r S(2) 4 )+ p 3 (S(3) r S(3) H )-»- ....} 

55 E{P f (S(1) r S(1 ) j_j) 2 + (P 2 (S(2) r S(2) 4 ) 2 + p 3 (S(3) r S(3) H ) 2 + ....} 

wherein Pi, P2, are coefficients. The equation defining the relevancy is not limited to the above and can be of any 
type which is selected depending on the user's preference. The scoring system can be predetermined. For example, if 
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user A is the user currently interacting with the device, user A in the memory is 1 .0, user B is 0.5, and another object is 
0.0. Similarly, if being touched is current contact, being touched is 1 .0, being stroked is 0.8, being hit is 0.2, and no con- 
tact is 0.0. Further, if the user is more important than contact, P for the user is larger than P for the contact The control 
system checks if the relevancy of the situation is higher than a predetermined level (step S3). If the relevancy of the sit- 

5 uation is higher than the predetermined level, the relevancy of the concerns is then determined (step S4). In an embod- 
iment, the relevancy of data stored in the memory can be determined as follows: The concern-generating unit 
comprises a plurality (p) of concern modules, each differently programmed to generate and output the concern-param- 
eter (C(n) ;) (n=l -p) , and, the best compensating event is selected if the relevancy calculated based on the difference 
between the initial variation (SC(n)-C(n))) and the recorded variation (ACfn)^) in each concern-parameter is higher 

10 than a p re-selected score, wherein the relevancy is defined by, for example, the following: 

Z((SC(n)-C(n)j)-AC(n) M ) 
rKSCfnJ-Cfn^-ACln)^ 2 

15 

HP^SCtlJ-COJj-ACO)^^ p 2 (SC(2) i -C(2) r AC(2) M )+ ....} 
£{p , (SC(1 )-C(1 ) r AC(1 ) M ) 2 + P 2 (SC(2) j-C(2) r AC(2) M ) 2 + ....} 

20 wherein p v Pg, ... are coefficients to weight some concern-parameters over the others. The equation defining the rele- 
vancy is not limited to the above and can be of any type which is selected depending on the user's preference. The con- 
trol system checks if the relevancy of the concerns is higher than a predetermined level (step S5). If the relevancy of 
the concerns is higher than the predetermined level, the data, especially on behavior, are extracted from the long-term 
memory (step S6). The device is actuated based on the information extracted from the long-term memory (step S7). A 

25 consequence of the device's behavior is sensed by the sensing unit to determine variations in the concerns as a result 
of the behavior, thereby updating the long-term memory (step S8). There are various ways to record the data on the 
event (explained later). In step S5, if the relevancy of the concerns is lower than the predetermined level, or if the rele- 
vancy of the situation is lower than the predetermined level (i.e., if no data on the event is extracted from the memory), 
the device can behave under predetermined rules (step S9). Such default behavior can simply be "explore" or "ran- 

30 dom". A consequence of the behavior by the device is recorded as a new event. In Figure 2, an event to best compen- 
sate for the variations in the concerns is found and suggests 'APPROACH" to increase the "PLAY" level. As a result, 
although an increase of 0.3 is expected, "PLAY" increases of 0.4. This difference is recorded in the long-term memory. 
[0054] In the above, if no event sufficiently relevant to the current event is found, the elements to define the situation 
can be restricted to one or more elements. For example, if user A is caressing the device, no situation may be found in 

35 the long-term memory. If the elements to define the situation are restricted to the "user" factor or if the "user" factor is 
highly weighted, the relevancy of the situation may satisfy the predetermined level. Further, if no specific situation is 
found in the memory, the control system checks if a superordinate situation can be found in the long-term memory to 
obtain a candidate behavior from the long-term memory. 

40 Object Recognition 

[0055] When the device has a new experience, that is, when no event is extracted from the long-term memory, the 
device can categorize an object or situation in order to record the new experience in the long-term memory. When the 
long-term memory suggest "user A" to play with, the device can recognize "user A", for example. User A can be defined 

45 by color, voice, face configuration, and/or name tag. Similarly, if the long-term memory suggests a "dark" and "quiet" 
place to "sleep", the place can be recognized. The term "dark" and "quiet" can be relative evaluation, and the device 
can find a relatively dark and quiet place among the objects the device is observing. The device is directed in a direction 
where values indicated by audio sensor and brightness sensor become lower. This relative evaluation can be performed 
easily. However, if the device observes a new object, the device cannot respond to it. For example, if "user A" is sug- 

50 gested, but the device does not observe user A in a predetermined period of time or until a change occurs in the con- 
cerns or emotions, an object the device observes can be selected as an object of concern. If the device does not 
observe user A but observes user B, the device may not recognize user B as an object of concern. If the objects are 
defined by genetic characteristics or loose rules such as a "moving" object making "sound", user A and user B can be 
an object of concern. After the device observes user B and while the device interacts with user B, the device can collect 

55 data on identification of user B by using the sensing unit The data can be stored in the long-term memory and written 
in the spreadsheet under the index of user #2. Subsequently, when the device observes user #2 inputs data on a data 
sheet in the working memory, the device extracts data on user #2 from the long-term memory. 
[0056] By using object-oriented languages such as JAVA and C++, the number of objects of concern can be 
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increased autonomously. For example, in JAVA, first, "objects" are defined by "parameters" and "methods", and then, 
instances are defined based on "parameter" and "method". Figure 8 is a diagram showing schematic processes of 
JAVA. In this figure, the "object" is "user", and "user parameters' are "color" and "voice", and "method" is 
(user parameter x (color + voice) ). Here, the program stores two instances; user #1 (color=-1 (red), voice=+1 (high)), 

5 and user #2 (co!or=*-1 (blue), voice=-1 (low)). When the device observes user #3 (color=0(yellow), voice=0(intermedi- 
ate)), the program checks whether user #3 falls within the profiles of user #1 or #2. Here, user #3 is neither red (- 1 ) nor 
blue (+1 ), indicating that user #3's color is new which is 0 (yellow). The program creates a new instance underthe index 
of user #3 and defines user #3 by color (0, yellow) and voice (0, intermediate). When the device does not observe an 
object of concern, the device may select an object which the device is seeing, as an object of concern as explained 

10 above. User #3 will be one of objects of concern. That is, when the concern-generating unit selects "play", the program 
checks whether the object the device is seeing is either one of users #1, #2, and #3. By continuing such experiments, 
the device stores many objects of concern. 

Emotion-Generating Unit 

15 

[0057] Emotions generated by the device may be used for modifying the behavior selected by extracting data from 
the long-term memory. The emotion-generating unit can be operated by various ways which include a neural network. 
Figure 9 is a diagram showing an embodiment of an emotion-generating unit using a neural network. The structure of 
the neural network in Figure 9 is similar to that indicated in Figure 6. The difference is that this neural network uses sig- 

20 nals from the concern-generating unit as one of the inputs. Alternatively, the output from the neural network can be mul- 
tiplied by signals from the concern-generating unit. The input-output relationship of the neural network may be 
regulated in advance by off-line training, although the neural network can be established off-line or on-line. If it is con- 
ducted on-line, output of the device is fed back to the control system, and coupling coefficients can be adjusted. Further, 
coupling coefficients can be modified using evolutionary computing technology such as genetic algorithms and genetic 

25 programming. However, if on-line establishment is conducted, a "training" period will be required. 

[0058] For establishing an emotion generation system, the technology disclosed in United States Patent Applica- 
tion No. 09/059,278, filed April 13, 1998, by Tamiya, et al., entitled "CONTROL SYSTEM FOR CONTROLLING 
OBJECT USING PSEUDO-EMOTIONS GENERATED IN THE OBJECT", can be adapted to the present invention. The 
reference is hereby incorporated herein as reference. In the above, each emotion-parameter is defined by a facial 

30 expression which the robot can output, and teacher data are not required to establish the neural network if output eval- 
uation can be conducted by using reward signals (e.g., being caressed) or penalty signals (e.g., being bit), instead of 
teacher data. 

[0059] Further, the emotion-generating unit can be constituted by a multi-dimensional map defined by the recog- 
nized signals and the emotion-parameters. 
35 [0060] For a very simple model, the input-output relationship in the emotion-generating unit can be simplified by 
directly relating recognized signals with emotions. For example, if the device is "hit", the device is "sad"; if the device is 
softly "touched", the device is "happy"; if the device is "low in power", the device is "angry"; if the device hears "loud 
sound", the device is "surprised", etc. 

40 Date Stored in Memory 

[0061] In the present invention, the device's behavior is selected based on the recorded variations in the concerns 
which may compensate for the current variations in the concerns. The variations in the concerns are extracted from the 
long-term memory. If the device has concern-parameters (C n ) initially; and predetermined standards for the concern 

45 levels are (SC), variations in its needs are (SC-C n ). If a situation relevant to the current situation is found in the long- 
term memory, the control system checks rf an event compensates for (SC-C n ). If information on variations in its con- 
cerns (ACVi) is found in the memory under the index of the situation ({AC* n _ 1 -(SC-C n ))> predetermined level), 
"behavior" is extracted from the memory. As a result, the device behaves accordingly (if there is no modification by emo- 
tions). If consequent variations in the concerns are AC n , the long-term memory can be updated in various ways. 

so [0062] Figure 1 0 is a diagram showing an embodiment of changes in emotions. AC n can be saved in the long-term 
memory in various ways and when the device next encounters the situation (#n+1), the variations of concerns are 
extracted from the long-term memory in the form of AC' n . rf the long-term memory has a plurality of columns for one 
situation and adds a new column for new data, AC' n can be: 

55 AC , n = LAC n /n 

AC n = E(AC n x 3L) / n {X is a time factor) 
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[0083] AC' n can be simply the average of AC n or the average of AC n multiplied by X. X is a coefficient which is a 
function of n so that the older the memory, the less the contribution becomes. If the long-term memory has one column 
for one situation, AC' n can be: 

5 AC n =AC n 

AC 4 n = aAC n + (1 -a) AC ^ (0<a<1) 

[0064] In the above, the long-term memory does not need to have a large memory capacity because one column 
10 is sufficient for one situation. The first equation is the same as the second equation when ce=1 . If o=1 , the precedent 
data will be abandoned and the data cannot be cumulative. If oc=0.1 -0.2, the data in the past can be used and modified 
effectively based on the current experience. The influence of the current experience is not large, and the device's 
behavior will change gradually. By changing a, the significance of the current experience can be adjusted. 
[0085] If the tendency of concern changes is considered to be important, AC' n can 
is be: 

AC n = AC n -AC n . 1 

[0086] Figure 11 is a diagram showing an embodiment of data storing processes under the rule, 
20 AC n = <xAC n + (1 -a) AC n . 1 (0<a< 1 ) . In the figure, the working memory has a data sheet 40a indicating LOVE -0.3, 
CURIOSITY -0.1, and HUNGER -0.1, as initial concerns (C n , wherein C n = C n1 , C^, ....). The long-term memory has 
a spreadsheet 41 indicating LOVE 0.4, CURIOSITY 0.0, and HUNGER 0.1, as variations in the concerns (ACVi), 
which update C n (data sheet 40b). The data sheet 40b indicates LOVE 0.1, CURIOSITY -0.1, and HUNGER 0.0, as 
modified concerns (C n + ACVi) (The modified concerns need not be simply (C n + AC^i) but can be a function of C n 
25 and AC^i)- To expect the modified concerns to be realized, the device is actuated and consequently, variations in the 
concerns occur (data sheet 40c). The data sheet 40c indicates LOVE 0.1 , CURIOSITY -0.1, and HUNGER -0.1 , as con- 
sequent concerns (C n .). The variations in the concerns after actuation of the device are AC n (AC n =C n .-C n ) . AC n is 
LOVE 0.4, CURIOSITY 0.0, and HUNGER 0.0. The rule, AC n = aAC n + (1-a) AC'^ , is used, wherein a=0.1. Vari- 
ations in the concerns to be saved in the long-term memory (AC n ) are LOVE 0.4, CURIOSITY 0.0, and HUNGER 0.09, 
30 which update the long-term memory. When variations in the concerns can be compensated for by AC n , the situation 
will be retrieved and the device will behave accordingly. 

Behavior-Planning Unit 

as [0067] Figure 12 is a diagram showing an embodiment of behavior patterns. As a result of variations in the con- 
cerns, behavior will be selected by finding an event to best compensate for the variations. The selected behavior can 
be modified by variations in its emotions. That is, the device's behavior is modified in accordance with the variations in 
the emotions. If the initial behavior is "APPROACH", when the device raises an "ANGRY" score, the behavior is 
changed to "ATTACK"; when the device raises a "FEAR" score, the behavior is changed to "AVOID"; when the device 

40 raises a "DISGUST" score, the behavior is changed to "RANDOM"; when the device raises a "SAD" score, the behavior 
is changed to "EXPLORE"; when the device raises a "HAPPY" score, the behavior is not changed, but may be more 
active (faster movement). If the initial behavior is "ATTACH", when the device raises a "HAPPY" score, the behavior is 
changed to "APPROACH"; when the device raises a "DISGUST score, the behavior is changed to "RANDOM"; and 
when the device raises an "ANGRY" score, the behavior is not changed, but may be more active (faster movement). 

45 When the device avoids the object, the device raises an "ANGRY" score (run away is not successful), the behavior is 
changed to "ATTACK". In the long-term memory, the action which is actually performed is recorded and linked with the 
other data under the index of the situation. 

[0068] These behavior patterns can be established by a neural network by regulating the input-output relationship 
or by a multi-dimensional map. Further, coupling coefficients of the neural network can be modified using evolutionary 
so computing technology such as genetic algorithms and genetic programming. However, if on-line establishment is con- 
ducted, a "training" period will be required. For establishing behavior patterns, the technology disclosed in United States 
Patent Application No. 09/059,278, filed April 13, 1998, by Tamiya, et al.„ as described above, can be adapted to the 
present invention. The reference is hereby incorporated herein as reference. 

[0069] Further, the behavior-planning unit can be constituted by a multi-dimensional map defined by the generated 
55 concern -parameters and the generated emotion-parameters. In the above, if the device has a plurality of moving parts, 
all moving parts can be allocated on a multi-dimensional map. This approach can be used in combination with a neural 
network, wherein a multi-dimensional map is connected downstream of the neural network. For example, sensed sig- 
nals, emotion-parameters, and concern-parameters (these data are stored in the working memory) are used as input 
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signals of the neural network. "APPROACH", "ATTACK", "EXPLORE", "AVOID", "RANDOM", "STILL", etc. are output- 
ted from the neural network. Training the neural network can be conducted off-line. Downstream of the neural network, 
a multi-dimensional map is connected, which regulates the output in such a way, for example, that if "APPROACH" 0.1 
and "EXPLORE" 0.3, the device slowly moves toward the object with a tail slowly moving. Not only the neural network 
5 but also the map can be modified by using evolutionary computing technology. Evolutionary reinforcement learning 
methods does not require teacher data. Evaluation functions can be obtained by using reward signals (e.g., being 
caressed) or penalty signals (e.g., being hit), instead of teacher data 

[0070] Figure 13 is a diagram showing an embodiment of a behavior control system. The behavior-planning unit 
can have a plurality of modules, each regulating one action. The modules can be selected by using technology 

w described above. Each module commands the device's movement sequentially on conditions of the external and inter- 
nal conditions. Figure 13 shows an embodiment of a "RUN AWAY" module. If the object is behind the device, the device 
moves forward (action 1). If the object is in front of the device, the device turns around (action 2). If predetermined con- 
ditions are satisfied, action 1 (moving forward) is triggered from action 2. A trigger may be defined by the distance from 
the object, the moving speed of the object and its direction, the intensity of sound the object is making, the absence of 

15 obstacles, etc. The trigger can be determined based on theoretical sum of all of the conditions, each having a trigger 
range. If the device moves a certain distance, the device stops (action 3). If the device stops, the device turns at 1 80 
degrees without condition (action 4). If the object is close to the device, action 2 is triggered. If the object is not close to 
the device, the series of actions end. In the above, each action is constituted by a sub-module. Each sub-module can 
include emotion-parameters. For example, the module for action 1 (moving forward) can be programmed to change the 

20 moving speed in accordance with emotion-parameters. If a "FEARFUL" score is high, the device moves faster than 
when the score is low. If the device moves away from the object, but the device is hit by the object, the "FEARFUL" score 
becomes high. These data are transferred to the working memory and update the data sheet. The data are saved in the 
long-term memory. Subsequently, when the device faces the same situation and extracts the data from the long-term 
memory, the sub-module for action 1 receives a signal to raise the moving speed (50). As described above, the spread- 

25 sheet in the long-term memory, the data sheet in the working memory, and the action modules can be modified based 
on the user's preferences. 

Other Features 

30 [0071] In the above, if the device does not have sufficient capacity (processing capacity, data storing capacity), the 
connected toy can be connected to an external system (computer) through cordless networks. That is, the data and/or 
programs used in the present system, including the long-term memory, the working memory, the emotion-generating 
unit, the concern-generating unit, and the behavior-planning unit, can be saved or installed in a separable medium such 
as a compact disc (ROM or RAM) or IC card, so that the user can implant the information in another device. In addition, 

35 the data and/or programs can be transmitted to another device via communication means. By using the above technol- 
ogy, plural memory media of plural devices can be hybridized or cross bred to create a new system. The data can be 
pooled from plural devices and installed into a new device which will possess an extensive memory even if the new 
device has never been used. By changing the programs, behavior patterns of the device can be changed. Further, any 
one or more of the intelligent portions of the device including the recognition unit, the emotion-generating unit, the con- 

40 cem-generating unit, and the behavior-planning unit can be installed in a main computer separately from the device, 
wherein a network is established between the main computer and the device via, for example, the Internet, so that the 
device can be made compact, and which may need to have simply sensing units, an output unit, and a communication 
unit for contacting the main computer. Through the sensing units of each device, it is possible to monitor the user or the 
external environment of each device, by use of the main computer. Further, each device can be connected to other 

45 devices to establish a network of devices to exchange information. 

Other Aspects 

[0072] In the present invention, correlations between various inputs and various outputs of the control system can 
so be determined using existing techniques such as neural networks, fuzzy neural networks, and genetic algorithms if the 
correlations are highly complex, or using existing techniques such as maps and functional equations if the correlations 
are rather simple. In this regard, Da Ruan (editor)" Intelligent Hybrid Systems - Fuzzy Logic, Neural Networks, and 
Genetic Algorithms -" Kluwer Academic Publishers (1 997), J.-S. R. Jang, C.-T. Sun, E. Mizutani,"Neuro-Fuzzy and Soft 
Computing" Prentice Hall Upper Saddle River, NJ 07458 (1997), C.-T. Lin and C. S. George Lee, "Neural Fuzzy Sys- 
55 terns" Prentice Hall Upper Saddle River, NJ 07458 (1998), and N. K. Kasabov, "Foundations of Neural Networks, Fuzzy 
Systems, and Knowledge Engineering" the MIT Press (1996) are hereby incorporated by reference. The above tech- 
niques can be combined, and learning control can be adapted for any technique. 

[0073] Further, in addition to genetic algorithms (GA), genetic programming (GP) or other evolutionary computing 
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techniques can be adapted to the present invention (Wolfgang Banzhaf, et al. (editor), "Genetic Programming, An Intro- 
duction - , pp. 363-377, 1999, Morgan Kaufman n Publishers, Inc., for example). These techniques are sometimes cate- 
gorized as "heuristic control" which includes evolution, simulated annealing, and reinforcement learning method (S. 
Suzuki, et al., "Vision-Based Learning for Real Robot Towards RoboCup", RoboCup - 97 Workshop, 23, 24, and 29 
5 August, 1997 Nagoya Congress Center, pp. 1 07-1 10; K. and Nurmela, et al., "Constructing Covering Designs By Sim- 
ulated Annealing", pp. 4-7, Helsinki University of Technology, Digital Systems Laboratory, Technical Reports No. 10, 
January 1 993, for example). These techniques can be adapted to the present invention without complication, based on 
the principle described earlier; that is, in the present invention, "evolutionary computing" includes the above various 
techniques. 

10 [0074] Further, the evolutionary computing includes a multi-agent system which is used for competitive co-evolution 
(Tamashiro, et al., "Study about the performance of competitive co-evolution in multi-agent system", Technical Report 
of the Institute of Electronics, Information and Communication Engineers, NS99-16 (1999-06), pp.37-41). 
[0075] Further, in the above, neural networks may be used for learning control rules. However, a CMAC (Cerebellar 
Model Arithmetic Computer) can also be used. The CMAC is excellent in terms of additional learning and the high 

is speed of learning, as compared with the hierarchical neural network. 

Other Applications 

[0076] In the above, the device may v be a pejgorjaJUabpt, t ov robot or ro bpJLpet However, the device of the present 
20 control system is not limited to a to y rob ot and can be any given control which can be used in a vehicle, an auxiliary 
drive of a bicycle, or wheelchair^or an industriaypbot. 

[0077] iFwffl be understood by those of skill in the art that numerous and various modifications can be made without 
departing from the spirit of the present invention. Therefore, it should be clearly understood that the forms of the present 
invention are illustrative only and are not intended to limit the scope of the present invention. 

25 

Claims 

1 . A method for adjusting behavior of a device based on the device's experience, said device comprising: (i) a sensing 
unit for sensing signals; (ii) a concern-generating unit programmed to generate concern-parameters; (iii) an actu- 

30 ating unit for actuating the device; and (iv) a memory storing a spreadsheet composed of multiple sets of data, each 
set corresponding to an event defined by signals received by the device, variations in its concern-parameters, and 
behavior of the device, 
said method comprising the steps of: 

35 (a) sensing signals receivable by the device; said device having concern-parameters (0\) (i=entry number or 

time) which vary from predetermined standards (SC); 

(b) extracting from the memory an event, if any, to best compensate for the variations in its concern-parame- 
ters; and 

(c) actuating the device based on the extracted information, and then computing variations in its concern- 
40 parameters (ACj), thereby updating the memory. 

2. The method according to Claim 1 , wherein the device further comprises an emotion-generating unit programmed 
to generate emotion-parameters, and wherein, in step (a), the device has variations in its emotion-parameters, and 
in step (c), behavior of the device extracted from the memory is modified by the variations in its emotion-parameters 

45 under predetermined rules. 

3. The method according to Claim 1 , wherein the updated memory stores variations in its concern-parameters (AC'j) 
linked with the event, wherein AC'j = ccACj + (1-a) AC' M (0<a<1) , wherein AC'|_i is the previous data saved 
under the index of the event. 

50 

4. The method according to Claim 1 , wherein, in step (b), if no data on the event is extracted from the memory, the 
device behaves under predetermined rules. 

5. The method according to Claim 1 , wherein the concern-generating unit comprises a plurality (p) of concern mod- 
55 ules, each differently programmed to generate and output the concern-parameter (C(n) j) (n=1 -p) , and, in step (b), 

the best compensating event is selected if the relevancy calculated based on the difference between the initial var- 
iation (SC(n)-C(n)j) and the recorded variation (AC'fn)^) in each concern-parameter is higher than a pre-selected 
score, wherein the relevancy is defined by r((SC(n)-C(n) j)-AC(n) M ) 2 (n=1-p) . 
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The method according to Claim 1 , wherein the input-output relationship of the concern-generating unit is predeter- 
mined, said concern-generating unit receiving p re-selected sensed signals and outputting the concern-parameters. 

The method according to Claim 1 , wherein the input-output relationship of the emotion-generating unit is predeter- 
mined, said emotion-generating unit receiving p re-selected sensed signals and the output of the concern-generat- 
ing unit, and outputting the emotion-parameters. 

The method according to Claim 1 , wherein the input-output relationship of the actuating unit is predetermined, said 
actuating unit receiving pre-selected sensed signals and the information from the memory, and actuating the 
device. 

The method according to Claim 1 , further comprising a working memory which temporarily pools and stores infor- 
mation from the sensing unit, the concern-generating unit, and the first-mentioned memory until the device com- 
pletes its action, and which outputs information to the concern-generating unit, the actuating unit, and the first- 
mentioned memory. 
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